Draft: Fix bytes and str issue #33
Conversation
|
In GitLab by @kinow on Jul 6, 2023, 14:59 mentioned in issue es/autosubmit#1075 |
|
In GitLab by @dbeltrankyl on Jul 6, 2023, 15:32 Oh, that is a problem. Do you know why we're setting this? I don't remember. The issue is that the commit directive doesn't necessarily have to be in the minimal.yml but doesn't need to be inside a file named expdef. So we should discard options a and b. I think we need to:
If you need to have this faster.. maybe we can go with the b) use minimal for now |
|
In GitLab by @kinow on Jul 7, 2023, 11:27 I think the idea was to save the commit as using only the URL and branch could point to a newer commit.
I think this is the cleanest solution 😥 I will see if I can maybe copy part of the code that retrieves the commit, or split the function into two (one fetches the commit, the other saves into the file). With these two functions, I should be able to use the fixed one that fetches the commit, and then we can work on the fix for the second part that saves it to the config file later. Thanks @dbeltrankyl ! |
|
In GitLab by @kinow on Feb 14, 2024, 13:28 p.s. probably for next week or Friday. I will rebase, check if this is still valid, and then write a test with pytest and update the MR 🎉 |
|
In GitLab by @dbeltrankyl on Mar 4, 2024, 13:21 mentioned in merge request es/autosubmit!394 |
|
In GitLab by @dbeltrankyl on May 7, 2024, 09:49 Maybe this needs an update? |
|
In GitLab by @kinow on May 7, 2024, 10:06 Definitely does, one of the many merge-requests I am pending to update, sorry. Will try to do it over next days. Thanks for the reminder, @dbeltrankyl ! 💪 |
|
In GitLab by @dbeltrankyl on Aug 23, 2024, 15:00 Hello @kinow , I see that this is unassigned ( although it could never be assigned) and it is an old merge. Do you need another person to take care of it? Maybe @egarriga or I can try to write the remaining stuff that I think is the pytest? |
|
In GitLab by @kinow on Aug 26, 2024, 08:58 Sounds good to me. @egarriga if you have spare time, you can have a look at this one. Otherwise, if I have some minutes by the end of a work day, I might have a stab at it and see if we can close this issue. Thanks @dbeltrankyl ! |
|
In GitLab by @egarriga on Aug 26, 2024, 09:52 I will take a look and work on it |
|
In GitLab by @kinow on Aug 26, 2024, 13:52 Thank you, Edgar! |
|
In GitLab by @kinow on Sep 5, 2024, 14:32 GitLab UI isn't very intuitive, IMHO (they could have copied GitHub's UI, then improved, instead of renaming pull-request/merge-request, moved things around, etc., IMHO... ). Opening the issue in AS linked above, you'll find that the issue is when you run " Looks like I found this when implementing the RO-Crate archive, where I had to test some parts of the code manually first. Hope that helps! |
|
In GitLab by @dbeltrankyl on Sep 16, 2024, 12:22 Removing the assigne |
|
In GitLab by @dbeltrankyl on Sep 16, 2024, 12:22 unassigned @egarriga |
|
In GitLab by @kinow on Sep 17, 2024, 14:05 Will manual test, and if that works with AS then will see if I can write a small unit test for this change. |
|
In GitLab by @kinow on Sep 17, 2024, 19:29 Confirmed that (autosubmit4) kinow@ranma:~/Development/python/workspace/autosubmit$ autosubmit -lc DEBUG clean --project a022
/home/kinow/Development/python/micromamba/envs/autosubmit4/bin/autosubmit:4: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
__import__('pkg_resources').require('autosubmit==4.1.10')
/home/kinow/Development/python/micromamba/envs/autosubmit4/lib/python3.9/site-packages/paramiko/pkey.py:100: CryptographyDeprecationWarning: TripleDES has been moved to cryptography.hazmat.decrepit.ciphers.algorithms.TripleDES and will be removed from this module in 48.0.0.
"cipher": algorithms.TripleDES,
/home/kinow/Development/python/micromamba/envs/autosubmit4/lib/python3.9/site-packages/paramiko/transport.py:259: CryptographyDeprecationWarning: TripleDES has been moved to cryptography.hazmat.decrepit.ciphers.algorithms.TripleDES and will be removed from this module in 48.0.0.
"class": algorithms.TripleDES,
Autosubmit admin user: eadmin is not set
Autosubmit is running with 4.1.10
Checking configuration files...
Expdef config file is correct
Platforms sections: OK
Jobs sections OK
Autosubmit general sections OK
Configuration files OK
Registering commit SHA...
Traceback (most recent call last):
File "/home/kinow/Development/python/workspace/autosubmit/autosubmit/autosubmit.py", line 2872, in clean
autosubmit_config.set_git_project_commit(autosubmit_config)
File "/home/kinow/Development/python/micromamba/envs/autosubmit4/lib/python3.9/site-packages/autosubmitconfigparser/config/configcommon.py", line 1910, in set_git_project_commit
Log.debug("Project branch is: " + project_branch)
TypeError: can only concatenate str (not "bytes") to str
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/kinow/Development/python/workspace/autosubmit/bin/autosubmit", line 59, in main
return_value = Autosubmit.parse_args()
File "/home/kinow/Development/python/workspace/autosubmit/autosubmit/autosubmit.py", line 725, in parse_args
return Autosubmit.clean(args.expid, args.project, args.plot, args.stats)
File "/home/kinow/Development/python/workspace/autosubmit/autosubmit/autosubmit.py", line 2888, in clean
raise AutosubmitCritical("Couldn't clean this experiment, check if you have the correct permissions", 7012,
log.log.AutosubmitCritical:
Trace: can only concatenate str (not "bytes") to str
[CRITICAL] Couldn't clean this experiment, check if you have the correct permissions [eCode=7012]
More info at https://autosubmit.readthedocs.io/en/master/troubleshooting/error-codes.html |
|
In GitLab by @kinow on Sep 17, 2024, 19:32 Hmmm, but then updating the version of the config parser in AS' Registering commit SHA...
Project branch is: master
Project commit SHA is: c2672568fb14706c6e66481bc7ceac467db8261e
Traceback (most recent call last):
File "/home/kinow/Development/python/workspace/autosubmit/autosubmit/autosubmit.py", line 2872, in clean
autosubmit_config.set_git_project_commit(autosubmit_config)
File "/home/kinow/Development/python/workspace/autosubmit4-config-parser/autosubmitconfigparser/config/configcommon.py", line 1929, in set_git_project_commit
content = open(self._exp_parser_file).read()
AttributeError: 'AutosubmitConfig' object has no attribute '_exp_parser_file'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/kinow/Development/python/workspace/autosubmit/bin/autosubmit", line 59, in main
return_value = Autosubmit.parse_args()
File "/home/kinow/Development/python/workspace/autosubmit/autosubmit/autosubmit.py", line 725, in parse_args
return Autosubmit.clean(args.expid, args.project, args.plot, args.stats)
File "/home/kinow/Development/python/workspace/autosubmit/autosubmit/autosubmit.py", line 2888, in clean
raise AutosubmitCritical("Couldn't clean this experiment, check if you have the correct permissions", 7012,
log.log.AutosubmitCritical:
Trace: 'AutosubmitConfig' object has no attribute '_exp_parser_file'
[CRITICAL] Couldn't clean this experiment, check if you have the correct permissions [eCode=7012]
More info at https://autosubmit.readthedocs.io/en/master/troubleshooting/error-codes.html
|
|
In GitLab by @kinow on Oct 10, 2024, 14:19 Commented on autosubmitconfigparser/config/configcommon.py line 1788 Not used in this project, nor in Autosubmit. |
|
In GitLab by @kinow on Oct 10, 2024, 14:19 Commented on autosubmitconfigparser/config/configcommon.py line 2155 Not used here, nor on Autosubmit. |
|
In GitLab by @kinow on Oct 10, 2024, 14:23 @dbeltrankyl I had a look at this issue today, but I am not sure how to fix it yet. I removed the only two other places where this However, in On these two functions, the idea seems to be write to files with the AS configuration. So While But what happens if the user has other files being loaded after version.yml and as_misc.yml? Isn't there a risk that these files might be loaded in other order, and the final values for the last command and version might be incorrect? And for other settings like the git project commit, would it be then the case to create |
In GitLab by @kinow on Jul 6, 2023, 14:58
@dbeltrankyl this fixes the
bytes andstr` issue, but now I get an error as it's trying to write to the old exp file.Should we a) check if the old exp file exists, b) use minimal, c) call some function that already does that for us?
Cheers